Improving Prototype Consistence for Wizard-of-oz Simulations and Evaluations

In this paper we describe a study of the technique of Wizard of Oz (WoZ) in perspectives of maintaining the simulation consistence for domestic communication evaluations. This study employs evaluation methodologies such like heuristic evaluations and video analysis to compare and review the consistence during WoZ experiments. In specific, we address this consistence study through rapid prototype designs and iterative evaluations. Series of prototypes were developed based on our domestic communication platform and these prototype features were compared across iterative evaluations. The consequences of our prototype development and experimental evaluations show that the role of the wizard is vital to the effectiveness of WoZ studies, and variables which relate to the wizard’s operations such like personal preferences and interaction paces need to be carefully addressed to improve the consistence of system simulations. The conclusions of this study are based on comparisons of different control panel designs for the wizard, which indicate that simulation consistence can be augmented through interfaces with specific flexibility and customisations.